A Personalized Talking/moving Head Presentation Using Image-based Transformations

This paper addresses the problem of multimedia presentation of a moving and talking head. Existing approaches either use a 3D geometrical representation or multiple 2D views to model and reconstruct faces. To achieve realistic appearance and personalization, and avoid complex models, we propose two approaches. First, we show how pure image-based view morphing can be used to create new moving and talking faces based on existing footage. These views form a video stream which in turn will be synchronized with the output of a personalized Text-To-Speech system. Then, to improve this approach, we use facial feature detection to fine tune the correlation-based optical flow computation used for view morphing, and also create talking views of a new person based on one non-talking view in any position and talking views of a reference person.